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Description 

RELATED APPLICATIONS 

5 [0001 ] The present application claims the benefit under 35 USC 1 1 9(e) of US provisional application 60/560,049 filed 
on April 8, 2004. 

FIELD OF THE INVENTION 

10 [0002] The invention relates to methods and apparatus for estimating a time to collision between a vehicle and an 
obstacle. 

BACKGROUND OF THE INVENTION 

15 [0003] Automotive accidents are a major cause of loss of life and dissipation of resources in substantially all societies 
in which automotive transportation is common. It is estimated that over 1 0,000,000 people are injured in traffic accidents 
annually worldwide and that of this number, about 3,000,000 people are severely injured and about 400,000 are killed. 
A report "The Economic Cost of Motor Vehicle Crashes 1994" by Lawrence J. Blincoe published by the United States 
National Highway Traffic Safety Administration estimates that motor vehicle crashes in the U.S. in 1994 caused about 

20 5.2 million nonfatal injuries, 40,000 fatal injuries and generated a total economic cost of about $1 50 billion. 

[0004] Lack of driver attention and tailgating is estimated to be a cause of about 90% of driver related accidents. 
Methods and apparatus that would alert a driver to a potential crash and provide him or her with sufficient time to 
undertake accident avoidance action would substantially moderate automotive accident rates. For example a 1 992 study 
by Daimler- Benz indicates that if passenger car drivers have a 0.5 second additional warning time of an impending rear 

25 end collision about 60 percent of such collisions can be prevented. An extra second of warning time would lead to a 
reduction of about 90 percent of rear-end collisions. 

[0005] Various systems collision warning/avoidance systems (CWAS) exist for recognizing an impending collision and 
warning a driver of the danger. US 5,529,138, describes a CWAS that uses a laser radar to determine distance and 
relative velocity to determine a time to collision of a vehicle with an object. US 5,646,612 describes a CWAS system 

30 comprising a laser radar and an infrared (IR) camera. A processor determines a time to collision (TTC) of a vehicle with 
an object responsive to signals provided by the laser radar and whether the object is a human, an animal or an inanimate 
object responsive to image data provided by the IR camera. The system operates to warn a driver of an impending 
collision with an object based on the TTC and kind of object "and properly performs deceleration and braking operations 
based on a position of the object and a speed of the vehicle is disclosed". 

35 [0006] US 2004/02241 6 discloses a vehicle warning and control system. The system having a video scanning system, 
and/or laser scanners which are mounted on the vehicle to scan the road in front of the vehicle. The system generates 
image information which is analyzed by a computer or in combination with a range sensing system to alert the driver of 
possible dangerous conditions during driving. 

[0007] Laser radar systems are relatively complicated systems that are generally expensive and tend to suffer from 
40 narrow field of view and relatively poor lateral resolution. PCT Publication WO 01 /3901 8 describes a CWAS that comprises 
a camera and a processor for processing image data provided by the camera. The camera provides images of an 
environment in which a vehicle is located and the processor determines a TTC of the vehicle with an obstacle by 
processing, optionally only, image data provided by the camera. The processor determines the TTC responsive to scale 
changes in the size of the obstacle as imaged in the images under the assumption that the relative velocity between the 
45 vehicle and the object is constant. 

SUMMARY OF THE INVENTION 

[0008] An aspect of some embodiments of the invention relates to providing an improved method and apparatus for 
50 determining at a given time t, a time to collision, TTC(t), of a vehicle with an object using a plurality of camera images 
of an environment in which the vehicle is located. 

[0009] An aspect of some embodiments of the invention relates to determining TTC(t) of the vehicle with the object 
by processing image data provided by the images without assuming that relative velocity between the vehicle and the 
object is substantially constant. In accordance with an embodiment of the invention, image data provided by the plurality 
55 of images is processed to provide an estimate of TTC(t), hereinafter T a (t), which is responsive to the relative acceleration 
between the vehicle and the object Optionally, only the image data is used to determine TTC(t). 

[0010] In accordance with an embodiment of the invention, to determine T a (t), the image data is processed to determine 
for the given time t, a ratio, hereinafter referred to as relative scale "S(t) M , between dimensions of a feature of the object 
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in different images of the plurality of the images. S(t) is used to determine an instantaneous relative velocity estimate 
for determining TTC(t), hereinafter T v (t), at the given time. T v (t) is equal to a distance between the vehicle and the object 
at time t divided by their instantaneous relative velocity. T v (t) is estimated from S(t), optionally using methods and 
algorithms described in PCT Publication WO 01/39018 cited above. According to an aspect of some embodiments of 
5 the invention, relative acceleration is expressed as a function of a time derivative T v (t) of T v (t) at a given time and T a 
(t) is determined as a function of the relative acceleration or a function of T v (t). 

[0011] An aspect of some embodiments of the invention relates to determining whether a vehicle is on a collision 
course with an object responsive to image data in a plurality of images of the vehicle environment that image the object. 
Only the image data is used to determine whether the objects are on a collision course. 

10 [0012] In accordance with an embodiment of the invention, the images are processed to determine trajectories for at 
least two features of the object toward which the vehicle is moving that substantially determine a width of the object 
parallel to the width of the vehicle. The vehicle and the object are determined to be on a collision course if, as the vehicle 
and object approach each other, for example as indicated by a value determined for TTC(t), the trajectories of the at 
least two features bracket at least a portion of the vehicle. In general, the object is another vehicle on the roadway on 

15 which the vehicle is moving and the at least two features, which may for example be edges, taillights or headlights of 
the other vehicle, are optionally features that determine a magnitude for the width of the other vehicle. 
[001 3] There is therefore provided in accordance with an embodiment of the present invention, a method of estimating 
a time to collision (TTC) of a vehicle with an object comprising : acquiring a plurality of images of the object; and determining 
a TTC from the images that is responsive to a relative velocity and relative acceleration between the vehicle and the 

20 object. The method comprises determining the relative velocity or a function thereof solely from information derived from 
the images and using the relative velocity or function thereof to determine TTC. 

[0014] Optionally, determining the relative velocity or function thereof, comprises determining a change in scale of an 
image of at least a portion of the object between images of the pluralities of images and using the change in scale to 
determine the relative velocity or function thereof. Additionally or alternatively the method comprises determining the 
25 relative acceleration or a function thereof from the images and using the relative acceleration or function thereof to 
determine TTC. Optionally, determining the relative acceleration or function thereof comprises determining a time de- 
rivative of the relative velocity or the function of the relative velocity. 

[0015] In some embodiments of the invention, the method comprises determining whether the vehicle and the object 
are on a course that leads to a collision at the TTC. Optionally, determining whether the vehicle and object are on a 
30 collision course comprises: determining motion of at least two features of the object relative to the vehicle from the 
images; and determining from the relative motions whether at TTC the first and second features straddle at least a part 
of the vehicle. 

[0016] There is further provided in accordance with a non-claimed embodiment of the invention, apparatus for deter- 
mining a time to collision (TTC) of a vehicle with an object comprising: at least one camera mounted in the vehicle and 

35 adapted for acquiring images of objects in the environment of the vehicle; and a processor that receives image data 
from the camera and processes the data to determine a TTC in accordance with a method of the invention. 
[0017] Optionally, the at least one camera comprises a single camera. Additionally or alternatively the apparatus 
comprises alarm apparatus for alerting a driver of the vehicle to a possible collision with the object responsive to the 
TTC. In some non-claimed embodiments of the invention, the apparatus comprises alarm apparatus for alerting persons 

40 outside of the vehicle to a possible collision of the vehicle with the object responsive to the TTC. In some non-claimed 
embodiments of the invention, the at least one camera images an environment in front of the vehicle. In some embodiments 
of the invention, the at least one camera images an environment in back of the vehicle. In some non-claimed embodiments 
of the invention, the at least one camera images an environment to a side of the vehicle. 

[0018] There is therefore provided in accordance with a non-claimed embodiment of the invention, a method of de- 
45 termining whether a first object and a second object are on a collision course comprising: acquiring an image of the 
second object from a position of the first object at each of a plurality of known times; determining motion of at least two 
features of the first object relative to the second object from the images; determining an estimate of a possible time to 
collision (TTC) of the first and second objects; and determining from the relative motions whether at the TTC, the first 
and second features straddle at least a part of the vehicle and if so that the objects are on a collision course. 
50 [001 9] Optionally, determining motion of the at least two features comprises determining lateral motion of the features 
relative to the first object. Optionally, determining whether the features straddle the first object at the TTC comprises 
extrapolating lateral locations of the features at TTC from their motion at times at which the images are acquired. 
Optionally, determining TTC comprises determining TTC from the images. In some embodiments of the invention TTC 
is determined only from the images. 
55 [0020] There is further provided in accordance with a non-claimed embodiment of the invention, a method of determining 
relative acceleration between a first and second object comprising: acquiring a plurality of images of the second object 
from locations of the first object; determining a change in scale of an image of at least a portion of the second object 
between images of the pluralities of images; using the change in scale to determine acceleration of a function of the 
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acceleration. Optionally, the acceleration or function thereof is determined only from data in the images. The scope of 
the invention is defined in the appended claims. 

BRIEF DESCRIPTION OF FIGURES 

[0021] Non-limiting examples of embodiments of the present invention are described below with reference to figures 
attached hereto, which are listed following this paragraph. In the figures, identical structures, elements or parts that 
appear in more than one figure are generally labeled with a same numeral in all the figures in which they appear. 
Dimensions of components and features shown in the figures are chosen for convenience and clarity of presentation 
and are not necessarily shown to scale. 

Fig. 1 schematically shows a first "following" vehicle having a collision warning/avoidance system (CWAS), operating 
to provide a warning of collision with a second "lead" vehicle in front of the following vehicle, in accordance with an 
embodiment of the present invention; 

Fig. 2 shows a graph that provides a comparison between TTC(t) for the vehicles shown in Fig. 1 determined equal 
to T v (t) in accordance with prior art and TTC(t) determined equal to T a (t), in accordance with an embodiment of the 
present invention; 

Fig. 3 shows a graph that compares the results of alerting the driver of the following vehicle to a possible collision 
with the lead vehicle in accordance with prior art and alerting the driver to a possible collision in accordance with 
the present invention; and 

Fig. 4 schematically illustrates determining whether a vehicle is on a collision course with another vehicle, in ac- 
cordance with an embodiment of the present invention. 

DETAILED DESCRIPTION OF EMBODIMENTS OF THE INVENTION 

[0022] Fig. 1 schematically shows a first, lead vehicle 21 traveling along a road 20 followed by a second, following 
vehicle 22 having a CWAS 30 in accordance with an embodiment of the present invention. CWAS 30 comprises a 
camera 32 that images, by way of example, the environment in front of following vehicle 22 and a processor 34 that 
processes image data provided by the camera to repeatedly update an estimate of a time to collision (TTC) of a possible 
rear end collision of following vehicle 22 with lead vehicle 21 . CWAS 30 comprises at least one device (not shown), for 
alerting a driver of following vehicle 22 to a possible collision with an object in front of the vehicle responsive to estimates 
of TTC provided by processor 34. 

[0023] At any given time t lead and following vehicles 21 and 22 are separated by a distance Z(t), hereinafter also 
referred to as "range", have a relative velocity V(t), which is changing with a relative acceleration a(t) (which may of 
course be zero). In accordance with an embodiment of the invention, at time t processor 34 determines an estimate of 
time to collision TTC(t) to be equal to T a (t), which is an estimate responsive to relative acceleration a(t), between lead 
vehicle 21 and following vehicle 22 at the given time. 

[0024] In accordance with an embodiment of the invention, processor 34 processes image data from a plurality of 
images to provide an estimate of relative scale S(t). Processor 34 determines S(t) from a ratio of dimensions of a feature 
of lead vehicle 21 in at least two different images of the lead vehicle acquired by camera 32 at times close to the given 
time. For example, assume that at first and second times t 1 and t 2 , which define a time interval At that optionally includes 
the given time t, camera 32 acquires first and second images respectively of lead vehicle 21. Let a distance between 
two features of lead vehicle 21 , or a dimension of a feature of the lead vehicle, such as for example width of the lead 
vehicle, have a length (as measured for example in pixels or millimeters) in the first and second images represented by 
w(t|) and w(t 2 ) respectively. Then, optionally, 



[0025] If at times t 1 and t 2 the lead and following vehicles 21 and 22 are separated by distances, i.e. "ranges", Z(t,) 
and Z(t 2 ) respectively, then assuming perspective projection of a scene imaged by camera 32 on a photosensitive surface 
of the camera 



S(t) = w(t 2 )/w(ti). 



(1) 



S(t) = Z(ti)/Z(t 2 ). 



(2) 



[0026] If the vehicles have an average relative velocity (assumed negative if distance between the vehicles is de- 
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creasing and positive if distance is increasing) V(t) during the time interval At, then assuming that At is relatively small, 
Z(t|) = [Z(t 2 ) - V(t)At] and S(t) may be written 

S(t)~[Z(t 2 )-V(t)At]/Z(t 2 ) 5 (3) 

from which it can be shown after relatively straightforward manipulation, 

Z(t 2 )/V(t) = -At/(S(t)-l). (4) 

[0027] Assuming that relative acceleration between lead and following vehicles 21 and 22 is zero (or that time to 
collision TTC(t) is independent of relative acceleration) TTC(t) for vehicles 21 and 22 may be estimated as equal to T v 
(t), where 

T v (t) = -Z(t)/V(t) = -Z(t 2 )/V(t) = At/(S(t)-l), (5) 

(it is recalled that V(t) is defined negative if the vehicles are getting closer to each other). 

[0028] The foregoing derivation of TTC(t) = T v (t), which assumes relative acceleration equal to zero and results in 
TTC(t) being dependent only on the instantaneous velocity, is based on the analysis presented in WO 01/39018 cited 
above. 

[0029] Abandoning the assumption of zero acceleration, in accordance with an embodiment of the present invention, 
TTC(t) is estimated as equal to T a (t), which has a value optionally determined responsive to an equation of the form, 

Z(t+T a (t)) = 0 = Z(t)+V(t)T a (t)+0.5a(t)T a (t)2. (6) 

Equation 6 assumes that from time t until collision, relative acceleration between lead and following vehicles 21 and 22 
is constant and equal to a(t) and that following time t by a time lapse equal to T a (t), range is equal to zero, i.e. the vehicles 
have contacted. 

[0030] Solving equation 6 for T a (t) provides an expression: 

T a (t) = (- V(t) + [V(t)2 - 2Z(t)a(t)]l/2 )/ a (t). (7) 

[0031] To evaluate T a (t) the inventors have noted that a time derivative T' v (t) of T v (t) may be written (note equation 5 
above), 

T'y(t) = d(-Z(t)/V(t))/dt = -Z'(t)/V(t) + Z(t) a(t)/V(t)2 = (a(t)Z(t)/V(t)2) -1 (8) 

It is convenient to define a parameter C(t), where 

C(t) = T' v (t)+1 = a(t)Z(t)/V(t)2 (9) 

from which, 



a(t) = C(t)V(t)2/Z(t). 



(10) 
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[0032] Substituting the expression for a(t) from equation 1 0 into the expression for T a (t) from equation 7, manipulating 
the results and using the expression T v (t) from equation 5 provides a "compact" expression for T a (t), namely 

T a (t) = [T v (t)/C(t)][l-(l+2C(t)]l/2. (1 1) 

[0033] In the above expressions, T v (t) is optionally approximated by -Z(t 2 )/V(t) = A/(S(t)-1) (equation 5 above). T v (t) 
is optionally determined by determining a time derivative responsive to values of T v (t) determined in accordance with 
to equation 5 for a plurality of different times t. For example, optionally, a plurality of values of T v (t) determined for a plurality 
of different times t, optionally before a given time t, are used to determine, using any of various methods known in the 
art, an analytic curve for T v (t) as a function of time. The time derivative of the analytic curve evaluated for the given time 
t provides T v (t) for the given time. 

[0034] To provide a numerical example that compares determining TTC(t) = T a (t), in accordance with an embodiment 

15 of the present invention, with determining TTC(t) in accordance with prior art in which TTC(t) = T v (t), assume that two 
vehicles are traveling at a same velocity equal to 70 kmph. Assume that the vehicles are separated by a range equal to 
50 m and that at a time t = 0 the "lead" driver of lead vehicle 21 spots an obstacle on road 20 and "hits" the brakes to 
decelerate at a maximum, constant, deceleration equal to 7.5 m/s 2 to stop the lead vehicle. Assume that at the time that 
the lead driver hits his or her brakes, the driver of following vehicle 22 has shifted his or her attention from the road in 

20 front of him or her and is looking at a road sign at the side of the road. As a result, the "following" driver does not notice 
the brake lights of lead vehicle 21 turning on at t = 0 when the lead driver hits the brakes or does not pay sufficient 
attention to the brake lights of the lead vehicle turning on. The driver of following vehicle 22 must rely on CWAS 30 to 
provide a warning of a possible rear end collision with lead vehicle 21 with sufficient lead-time to prevent the collision. 
Finally, assume that when alerted, the "following" driver applies the brakes to decelerate following vehicle 22, also at a 

25 constant deceleration of 7.5 m/s 2 and that from a time at which the following driver is alerted to a danger there is a lag 
reaction time of about 1 .6 seconds until the driver effectively applies the brakes of the following vehicle. (Driver reaction 
times are noted on page 27 of "Vehicle and Infrastructure-Based Technology for the Prevention of Rear-End Collisions"; 
Special Investigation Report, No. PB2001 -91 7003, published by the National Transportation Safety Board, Washington 
D.C. May 1, 2001. On page 27 the report notes that "typical driver perception-reaction time ranges from 0.9 to 2.1 

30 seconds with the 95-th percentile reaction time of 1 .6 seconds") 

[0035] Fig. 2 shows a graph 40 that provides a comparison between TTC(t) determined equal to T v (t) and TTC(t) 
determined equal to T a (t) in accordance with an embodiment of the present invention, subject to the assumptions 
described in the preceding paragraph. Curves 41 and 42 give values of T a (t) and T v (t) noted along the left hand ordinate 
of graph 40 as functions of time noted along the abscissa of the graph from the time t = 0 at which lead driver of lead 

35 vehicle 21 applies the brakes. By way of example, it is assumed that CWAS 30 activates an alarm to alert a driver of 
following vehicle 22 to a possible collision if its evaluated TTC(t) is equal to or less than a collision alarm time (CAT) of 
about 2.8 seconds. CAT equal to 2.8 seconds is indicated in graph 40 by a line 44. From curve 41 and CAT line 44 it is 
seen that CWAS 30 alerts the driver of following vehicle 22 to a possible rear end collision with lead vehicle 21, in 
accordance with an embodiment of the invention, about 0.85 seconds after the driver of the lead vehicle has applied the 

^o brakes. 

[0036] Because of the 1.6 seconds lag in reaction time, the following driver manages to apply the brakes only at a 
time 2.45 seconds after the lead driver applies the brakes to lead vehicle 21. An arrow 46 indicates the elapsed time 
between the time at which the alert is given in accordance with an embodiment of the invention and a time at which the 
following driver applies the brakes. The discontinuity in T a (t) occurs at a time at which the following driver applies the 

45 brakes and for a short period of time while lead vehicle 21 is still decelerating and the lead vehicle has not come to a 
full stop, the relative acceleration is zero. Similarly, from curve 42 it is seen that were CWAS 30 to alert the driver in 
accordance with prior art, i.e. TTC(t) = T v (t), the following driver would be alerted to a possible collision at a time about 
1 .85 seconds after the driver of lead vehicle 21 applied the brakes. The alert provided by prior art is given almost a full 
second later than the alert provided by an embodiment of the invention and the following driver would only apply the 

so brakes at a time of about 3.45 seconds after the lead driver applies the brakes. An arrow 48 indicates the elapsed time 
between the time at which the alert is given in accordance with prior art and a time at which the following driver applies 
the brakes. 

[0037] The import of the added warning time afforded the driver by an embodiment of the present invention is that the 
driver of following vehicle 22 avoids a collision with lead vehicle 21 that the driver would not avoid given an alert based 
55 on TTC(t) = T v (t). 

[0038] Fig. 3 shows a graph 50 that compares the results of alerting the driver in accordance with an embodiment of 
the present invention, i.e. TTC(t) = T a (t), for CAT = 2.8 seconds with results of alerting the driver in accordance with 
prior art i.e. TTC(t) = T v (t) for the same CAT. Curve 51 , also labeled Z a (t) in an upper portion 54 of graph 50 gives range 
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between lead vehicle 21 and following vehicle 22 as a function of time after the driver in lead vehicle 21 applies the 
brakes for the case in which the driver of following vehicle 22 applies the brakes after being alerted by CWAS 30, in 
accordance with an embodiment of the invention. Curve 61 , also labeled V a (t), in a bottom part 64 of graph 50, corresponds 
to curve 51 and gives the relative velocity between lead and following vehicle 21 and 22 for the case where the driver 

5 of following vehicle 22 applies the brakes responsive to an alert in accordance with the invention. Curve 52, also labeled 
Z v (t), in upper portion 54 of graph 50 gives range between lead vehicle 21 and following vehicle 22 were the driver in 
following vehicle 22 to apply the brakes responsive to an alert in accordance with the prior art. Curve 62 in bottom part 
64 corresponds to curve 52 and gives the relative velocity V v (t) between the lead and following vehicles were the driver 
of following vehicle 22 to apply the brakes responsive to an alert based on the prior art. 

10 [0039] Curve 51 shows that range Z a (t) between lead and following vehicles never reaches zero, but instead both 
vehicles come to a full stop with a range between the vehicles equal to about 0.4 m at a time equal to about 5.2 seconds 
after the lead driver applies the brakes. Curve 61 shows that relative velocity V a (t), which is equal to zero before the 
driver of lead vehicle 21 applies the brakes (both lead and following vehicles 21 and 22 are traveling at a same velocity), 
decreases rapidly during a period in which lead vehicle 21 is decelerating after being braked until a time at which the 

*5 driver of following vehicle 22 manages to apply the brakes. Thereafter, for a short time, until lead vehicle 21 comes to 
a full stop, relative velocity is constant while both vehicles lead and following vehicles 21 and 22 decelerate at a same 
acceleration (7.5 m/s 2 ) and relative acceleration is zero. After lead vehicle 21 comes to a stop at a time indicated by an 
arrow witness line 69 also labeled with the word "STOP", the relative velocity increases rapidly to zero as deceleration 
of following vehicle 22 provides a positive relative acceleration. 

20 [0040] Curves 52 and 62 indicate a substantially different scenario than curves 51 and 61 . Curve 51 shows that range 
Z v (t) crosses zero and following vehicle 22 "meets" lead vehicle 21 at a time equal to about 4 seconds indicated by an 
arrow witness line 71 , also labeled "CRASH". Curve 62 shows that at the time that the vehicles meet, the magnitude of 
relative acceleration V v (t) is quite large, indicating that following vehicle 22 does not contact lead vehicle 21 gently, but 
crashes into lead vehicle with substantial force. It is noted that whereas in the above described scenario, sufficient 

25 warning is provided by CWAS 30 to prevent a crash, a warning in accordance with an embodiment of the invention, if 
not sufficient to prevent a crash, will in general provide relatively more time to mitigate severity of a crash. 
[0041 ] Whereas a method in accordance with an embodiment the invention for determining TTC(t) in accordance with 
T a (t) can provide an improved determination of TTC(t), it does not by itself determine whether, if no action is taken by 
a driver, a collision will actually occur. For example, a lead vehicle may be located in a driving lane adjacent to that in 

30 which a following vehicle is located. A CWAS in the following vehicle, using only a method similar to that described 
above, may determine that the following vehicle will rear-end the lead vehicle at a particular TTC, when in fact the 
following vehicle is not on a collision course with the lead vehicle but will just pass the lead vehicle at the particular TTC. 
[0042] In accordance with an embodiment of the invention, a CWAS installed in a vehicle processes images provided 
by its camera not only to determine a TTC for the vehicle with an object, but also to determine whether the object and 

35 the vehicle are on a collision course. In accordance with an embodiment of the invention, the CWAS's processor deter- 
mines trajectories for at least two features of an object with which the vehicle is closing that substantially determine a 
width of the object parallel to the width of the vehicle. The CWAS determines that the vehicle and the object are on a 
collision course if, as the vehicle and object approach each other, for example as indicated by TTC(t) = T a (t), the 
trajectories of the at least two features bracket at least a portion of the vehicle comprising the CWAS. Usually, the object 

40 is another vehicle on the roadway on which the vehicle comprising the CWAS is moving and the at least two features, 
which may for example be edges, taillights or headlights of the other vehicle, are optionally features that determine a 
magnitude of the width of the other vehicle. 

[0043] Fig. 4 is a schematic birds-eye view of lead and following vehicles 21 and 22 on road 20 shown in Fig. 1 and 
illustrates a situation in which a CWAS, e.g. CWAS 30, in accordance with an embodiment of the invention operates to 

45 determine if the two vehicles are on a collision course. 

[0044] It is assumed, by way of example, that road 20 is a two lane highway that curves to the left and that lead vehicle 
21 is in a right hand lane 61 and following vehicle 22 is in a left hand passing lane 62. Lanes 61 and 62 are separated 
by lane markings 63. Following vehicle 22 is accelerating, or has accelerated, to a passing velocity in order to pass lead 
vehicle 22 and is schematically shown in dashed lines at three different locations on highway 20 relative to lead vehicle 

50 21 as the following vehicle passes the lead vehicle. Optionally, CWAS 34 is operating to update TTC(t) for following and 
passing vehicles 21 and 22 in accordance with an embodiment of the invention and determines TTC(t) = T a (t). 
[0045] In accordance with an embodiment of the invention, processor 34 processes images provided by camera 32 
to identify and locate at least two features that determine a width of lead vehicle 21 in each of a plurality of the images. 
Any of many various pattern recognition algorithms known in the art may be used to identify the features. For example, 

55 an edge detection algorithm may be used to identify edges 64A and 66B of lead vehicle 22 or taillights 66A and 66B 
that are separated by a distance substantially equal to the width of the vehicle. By way of example, it is assumed that 
processor 34 identifies and locates taillights 66A and 66B of lead vehicle 22 in each of a plurality of images as following 
vehicle 22 passes lead vehicle 21 . 
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[0046] In Fig. 4 lead vehicle 21 is schematically shown in an image 70 acquired by camera 32 at each of the positions 
of following vehicle 22 shown in the figure. For each position of vehicle 21 , image 70 acquired at the position is shown 
immediately to the left of the vehicle. Features in images 70 are optionally located relative to an image x-y coordinate 
system having a center 72 located at a point in the images corresponding to the optic axis of camera 32. As following 
5 vehicle 22 draws near to lead vehicle 21, "taillight images" 66A' and 66B' of taillights 66A and 66B respectively move 
progressively to the right of center 72 along the x-axis. In accordance with an embodiment of the invention, processor 
34 processes images 70 to determine whether motion of taillight images 66A' and 66B' along the x-axis of images 
acquired by camera 32 indicate whether vehicles 21 and 22 are on a collision course. 

[0047] Letthex-coordinates taillight images 66A' and 66B' in each of the images acquired by camera 32 be represented 
to by x a (t) and x b (t) and let corresponding real space x-coordinates of taillights 66A and 66B relative to the location of 
camera 32 in vehicle 21 be respectively X A (t) and X B (t). For convenience of presentation the real space x-coordinate, 
X c , of camera 32 is defined equal to zero (i.e. the camera is at the origin of coordinates). At some initial time, t D , at which 
a first image 70 of lead vehicle 21 is acquired by camera 32, the x-coordinates of taillight images 66A' and 66B' are x a 
(t Q ) and x b (t Q ) and let the range at time to of the lead vehicle relative to following vehicle 22 be Z(t Q ). Using perspective 
15 projection it can be shown that the range Z(t) of lead vehicle 21 at a time t later than t Q may be expressed, 



Z(t) = [x a (t) - x b (t)] Z(t 0 )/[x a (t 0 ) - x b (to)]. 

20 

Using equation 12, the real space x-coordinates of taillights 66A and 66B may be written, 



X A (t) = (x a (t)Z(t 0 )/f)([x a (t 0 ) - x b (t 0 )]Z(t 0 )/[x a (t) - x b (t)]) (13) 

25 

X B (t) = (x b (t)Z(t 0 )/f)([x a (t 0 ) - x b (t 0 )]Z(to)/[x a (t) - x b (t)]), (14) 

30 where f is the focal length of camera 32. 

[0048] In accordance with an embodiment of the invention, processor 34 processes image data provided by camera 
32 to determine values for x a (tj) and x b (t,) and therefrom X A (t,) and X B (t,) responsive to equations 1 3 and 1 4 at a plurality 
of times t| equal to and greater than to. At a given time t, the processor extrapolates the determined values for X A (tj) and 
X B (tj) to provide values for X A (TTC(t)) and X B (TTC(t)). Optionally, TTC(t) = T a (t). In accordance with an embodiment of 

35 the invention, if X A (TTC(t)) and X B (TTC(t)) straddle the coordinate X c of camera 32 (i. e. have opposite signs assuming 
X c = 0) then processor 32 determines that lead and following vehicles 21 and 22 are on a collision course. 
[0049] It is noted that since a sufficient condition for X A (TTC(t)) and X B (TTC(t)) to straddle X c is that they have opposite 
signs, processor 32 can use an arbitrary value for Z(t Q ) when determining if they straddle X c . However, if both X A (TTC 
(t)) and X B (TTC(t)) lie to the left or the right of X c , processor 32 cannot determine for sure, responsive only to equations 

40 13 and 14 if lead and following vehicles 21 and 22 are, or are not, on a collision course without a realistic value for Z 
(t Q ). For a given set of values for x a (tj) and x b (tj), Z(t Q ) determines magnitudes of displacement of X A (TTC(t)) and X B 
(TTC(t)) from X c . In particular, if both X A (TTC(t)) and X B (TTC(t)) are displaced to a same, one side of X c , Z(t Q ) determines 
if they are displaced sufficiently so that vehicles do not collide. In some embodiments of the invention, processor 34 
processes images 70 using methods described in "Vision Based ACC with a single Camera: Bounds on Range and 

45 Range Rate Accuracy"; G.P. Stein, O. Mano and A. Shashua; Proceedings of IEEE Intelligent Vehicles Symposium 
(IV2003), pages 120-125, Jun. 9-11, 2003, Columbus, OH. USA; to determine a value for Z(t Q ). For relatively short 
ranges up to about 20 to 30 meters motion parallax may optionally be used to determine a value for Z(t 0 ). 
[0050] Whereas in the above description of exemplary embodiments of the invention a CWAS was installed in the 
front end of a vehicle to alert the vehicle's driver to a possible collision with an object in front of the vehicle, a CWAS in 

so accordance with an embodiment of the invention may of course be installed elsewhere in a vehicle. For example, a 
CWAS may be installed in the rear of a vehicle to alert the driver to a possible rear end collision or in the sides of the 
vehicle to alert the driver to possible side collisions. A CWAS installed in such locations of a vehicle may provide a driver 
with sufficient time to enable him to take action that might mitigate severity of a rear end or side collision. 
[0051] A CWAS in accordance with an embodiment of the invention may operate any of various alarms, for example 

55 audio, visual or tactile alarms, to alert a driver to a possible collision. However, it is noted that a possible collision between 
a vehicle comprising a CWAS and another vehicle, will in general have potential to affect more than the driver and 
occupants of the vehicle outfitted with the CWAS. The possible collision does of course have substantial potential to 
affect the driver and occupants of the other vehicle and persons in the immediate environment of the vehicles. Further- 
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more, were the driver of the other vehicle and persons in the immediate environment made aware of the possible collision 
in which they may be participants, they might be able to take action that contributes to avoiding the collision or mitigating 
its effects. 

[0052] Therefore, in accordance with some embodiments of the invention, a CWAS is configured to alert persons 
5 other than the driver of the vehicle in which it is installed to a potential collision. When a possible collision is anticipated 
by the CWAS it optionally operates an alarm or alarms that alert drivers of other vehicles and pedestrians in the envi- 
ronment of the vehicle to the possible collision. For example, the CWAS may control the vehicle's horn to generate a 
particular type of audio alarm or the vehicles lights to flash warning signals. 

[0053] It is noted that whereas in the exemplary embodiments, a CWAS is described as processing images provided 
10 by its camera to determine whether to alert a driver to a potential collision, a CWAS in accordance with an embodiment 
of the invention may process data additional to image data to determine risk of a potential collision. For example, the 
CWAS may use data provided by a vehicle's speedometer, or sensors that generate signals responsive to operation of 
the vehicle's brakes or gas pedal to determine risk of a collision. 

[0054] In addition, a CWAS in accordance with some embodiments of the invention may perform functions other than 
15 to warn a driver and optionally other persons of an impending collision. For example, if the CWAS determines responsive 
to a TTC that risk of a collision is greater than a predetermined risk level and that driver is not undertaking any collision 
avoidance action, the CWAS may be equipped to apply the brakes. 

[0055] In the description and claims of the present application, each of the verbs, "comprise" "include" and "have", 
and conjugates thereof, are used to indicate that the object or objects of the verb are not necessarily a complete listing 

20 of members, components, elements or parts of the subject or subjects of the verb. 

[0056] The present invention has been described using detailed descriptions of embodiments thereof that are provided 
by way of example and are not intended to limit the scope of the invention. The described embodiments comprise different 
features, not all of which are required in all embodiments of the invention. Some embodiments of the present invention 
utilize only some of the features or possible combinations of the features. Variations of embodiments of the present 

25 invention that are described and embodiments of the present invention comprising different combinations of features 
noted in the described embodiments will occur to persons of the art. The scope of the invention is limited only by the 
following claims. 



30 Claims 

1 . A method of estimating a time-to-collision, TTC, of a vehicle with an object comprising the steps of: 

(a) acquiring a plurality of images of the object at known time intervals; and characterized by: 
35 (b) determining the time-to-collision, TTC, solely from information derived from the images and the time intervals, 

wherein determining the TTC is based on a relative velocity and relative acceleration between the vehicle and 
the object. 

2. The method according to claim 1 , wherein 

40 the relative velocity is determined from the images. 

3. The method according to claim 1 , wherein said step b for determining the time-to-collision, TTC, includes determining 
a change in scale of an image of at least a portion of the object and using the change in scale for determining a 
function of the relative velocity. 

45 

4. The method, according to claim 1 , 

wherein a function of the relative acceleration is determined from the images. 

5. The method according to claim 4, wherein said function of the relative acceleration includes determining a time 
50 derivative of a function of the relative velocity, 

6. The method according to claim 3, wherein determining a change in scale includes determining a ratio between a 
dimension of the object in a first one of the images and the same dimension of the object in a second one of the images. 

55 7. The method according to claim 6, wherein said determining a function of the relative velocity includes determining 
a function Tv = [1/(S-1)]AT where S is the ratio and AT is a time lapse between two images of the images. 

8. The method according to claim 7, wherein a function of the relative acceleration is determined based on a time 
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derivative TV, of function Tv. 

9. The method according to claim 8, wherein the TTC is determined responsive to a function of Tv and TV. 

10. The method according to claim 8, wherein TTC is determined responsive to the expression: TTC (t) = [Tv/C)] 
[1 -(1 +2C)] 1/2 , where C = TV+1 . 

1 1 . The method according to claim 1 , further comprising the step of: 

(c) determining whether the vehicle and the object are on a course that leads to a collision at the TTC. 

12. A method according to claim 1 1 , wherein said determining whether the vehicle and object are on a course that leads 
to a collision at the TTC includes: 

(a) determining respective motions of at least two features of the object relative to the vehicle from the images; and 

(b) determining from the relative motions whether at TTC said at least two features straddle at least a part of 
the vehicle. 

13. The method, according to claim 1 , further comprising the step of: 

(c) alerting a driver of the vehicle of a possible collision with the object responsive to the TTC. 

14. The method, according to claim 1 , further comprising the step of: 

(c) alerting based on the TTC, at least one person outside of the vehicle of a possible collision of the vehicle 
with the object. 



Patentanspruche 

1. Ein Verfahren zur Abschatzung einer Zeit bis zur Kollision, TTC, eines Fahrzeugs mit einem Gegenstand, das die 
folgenden Schritte umfasst: 

a) Aufzeichnen einer Vielzahl von Bildern des Gegenstands bei bekannten Zeitintervallen; und gekennzeichnet 
durch: 

b) Bestimmen der Zeit bis zur Kollision, TTC, alleine aus der Information, die von den Bildern abgeleitet wird 
und den Zeitintervallen, worin die Bestimmung der TTC auf einer relativen Geschwindigkeit und einer relativen 
Beschleunigung zwischen dem Fahrzeug und dem Gegenstand basiert. 

2. Das Verfahren gemaB Anspruch 1 , worin die relative Geschwindigkeit aus den Bildern bestimmt wird. 

3. Das Verfahren gemaB Anspruch 1 , worin der Schritt b) zur Bestimmung der Zeit bis zur Kollision, TTC, die Bestim- 
mung einer Anderung in der Skalierung eines Bildes einschlieBt, von mindestens einem Teil des Gegenstands, und 
die Verwendung der Anderung in der Skalierung zur Bestimmung einer Funktion der relativen Geschwindigkeit. 

4. Das Verfahren gemaB Anspruch 1 , worin eine Funktion der relativen Beschleunigung aus den Bildern bestimmt wird. 

5. Das Verfahren gemaB Anspruch 4, worin die Funktion der relativen Beschleunigung die Bestimmung einer Zeit- 
Ableitung einer Funktion der relativen Geschwindigkeit einschlieBt. 

6. Das Verfahren gemaB Anspruch 3, worin die Bestimmung einer Anderung in der Skalierung die Bestimmung eines 
Verhaltnisses zwischen einer Abmessung des Gegenstands in einem ersten der Bilder und derselben Abmessung 
des Gegenstands in einem zweiten der Bilder einschlieBt. 

7. Das Verfahren gemaB Anspruch 6, worin die Bestimmung einer Funktion der relativen Geschwindigkeit die Bestim- 
mung einer Funktion Tv = [1/(S-1)]AT einschlieBt, worin S das Verhaltnis ist und AT ist eine Zeitspanne zwischen 
zwei Bildern der Bilder. 
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8. Das Verfahren gemaB Anspruch 7, worin eine Funktion der relativen Beschleunigung bestimmt wird, basierend auf 
einer Zeitableitung TV der Funktion Tv. 

9. Das Verfahren gemaB Anspruch 8, worin die TTC bestimmt wird als Reaktion auf eine Funktion von Tv und TV. 

10. Das Verfahren gemaB Anspruch 8, worin die TTC bestimmt wird als Reaktion auf den Ausdruck: TTC(t)=[Tv/C)] 
[1-(1+2C)] 1/2 , worin C=TV+1. 

1 1 . Das Verfahren gemaB Anspruch 1 , das weiter der folgenden Schritt umfasst: 

c) Bestimmen, ob das Fahrzeug und der Gegenstand auf einer Bahn sind, die zu einer Kollision zu der TTC fuhrt. 

12. Ein Verfahren gemaB Anspruch 1 1 , worin die Bestimmung, ob das Fahrzeug und der Gegenstand auf einer Bahn 
sind, die zu einer Kollision zu der TTC fuhrt, folgendes einschlieBt: 

a) Bestimmen von jeweiligen Bewegungen von mindestens zwei Merkmalen des Gegenstandes relativ zu dem 
Fahrzeug aus den Bildern; und 

b) Bestimmen aus den relativen Bewegungen, ob bei der TTC die mindestens zwei Merkmale mindestens einen 
Teil des Fahrzeugs uberspannen. 

1 3. Das Verfahren gemaB Anspruch 1 , das weiter den folgenden Schritt umfasst: 

c) Warnen eines Fahrers des Fahrzeugs vor einer moglichen Kollision mit dem Gegenstand als Reaktion auf 
die TTC. 

14. Das Verfahren gemaB Anspruch 1, das weiter den folgenden Schritt umfasst: 

c) Warnen, basierend auf der TTC, von mindestens einer Person auBerhalb des Fahrzeugs vor einer moglichen 
Kollision des Fahrzeugs mit dem Gegenstand. 



Revendications 

1. Procede d'estimation du temps de collision, TTC (time-to-collision), d'un vehicule avec un objet comprenant les 
etapes consistant a : 

(a) acquerir une pluralite d'images de I'objet a 5des intervalles de temps connus ; et caracterise par : 

(b) determiner le temps de collision, TTC, exclusivement a partir d'informations derivees des images et des 
intervalles de temps, dans lequel la determination du TTC est base sur une velocite relative et une acceleration 
relative entre le vehicule et I'objet. 

2. Procede selon la revendication 1 , dans lequel 

la velocite relative est determinee a partir des images. 

3. Procede selon la revendication 1 , dans lequel ladite etape b de determination du temps de collision, TTC, comprend 
la determination d'un changementd'echelled'une imaged'au moins une partiede I'objet etl'utilisationduchangement 
d'echelle pour determiner une fonction de la velocite relative. 

4. Procede selon la revendication 1 , 

dans lequel une fonction de I'acceleration relative est determinee a partir des images. 

5. Procede selon la revendication 4, dans lequel ladite fonction de I'acceleration relative comprend la determination 
d'une duree derivee d'une fonction de la velocite relative. 

6. Procede selon la revendication 3, dans lequel la determination d'un changement d'echelle comprend la determination 
d'un rapport entre une dimension de I'objet dans une premiere parmi les images et la meme dimension de I'objet 
dans une seconde parmi les images. 
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7. Procede selon la revendication 6, dans lequel ladite determination d'une fonction de la velocite relative comprend 
la determination d'une fonction Tv = [1/(S-1)]AT ou S est le rapport et AT est I'intervalle de temps entre deux images 
parmi les images. 

8. Procede selon la revendication 7, dans lequel une fonction de I'acceleration relative est determinee en se basant 
sur une duree derivee TV, de la fonction Tv. 

9. Procede selon la revendication 8, dans lequel le TTC est determine en reponse a une fonction de Tv et TV. 

10. Procede selon la revendication 8, dans lequel le TTC est determine en reponse a I'expression : TTC (t) = [Tv/C] 
[1-(1+2C)] 1/2 , ou C = T'v+1. 

11. Procede selon la revendication 1, comprenant en outre I'etape consistant a : 

(c) determiner si le vehicule et I'objet se trouvent sur une trajectoire conduisant a une collision au moment du TTC. 

12. Procede selon la revendication 1 1 , dans lequel I'etape de determination si le vehicule et I'objet se trouvent sur une 
trajectoire conduisant a une collision au moment du TTC comprend : 

(a) ladetermination de mouvements respectifs d'au moins deux caracteristiques de I'objet par rapport au vehicule 
a partir des images ; et 

(b) la determination a partir des mouvements relatifs si au moment du TTC lesdites au moins deux caracteris- 
tiques heurtent au moins une partie du vehicule. 

13. Procede selon la revendication 1 , comprenant en outre I'etape consistant a : 

(c) alerter un conducteur du vehicule d'une collision possible avec I'objet en reponse au TTC. 

14. Procede selon la revendication 1 , comprenant en outre I'etape consistant a : 

(c) alerter, en se basant sur le TTC, au moins une personne a I'exterieur du vehicule d'une collision possible 
du vehicule avec I'objet. 
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